Results

cesm2.ssp245

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric naive cnn xgboost lstm
diff_of_means 2.27 3.70 4.58 10.94
ratio_of_sd 0.92 0.91 0.91 0.82
monthly_amplitude_ratio_of_means 0.91 0.90 0.88 0.72
ks_mean_on_coarse_res_with_extremes 0.33 0.22 0.15 0.45
qqplot_mae 8.15 10.15 11.01 24.86
acf_mae 0.04 0.03 0.01 0.08
extremogram_mae 0.04 0.04 0.03 0.05

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

cesm2.ssp370

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric naive cnn xgboost lstm
diff_of_means 3.38 4.54 5.80 11.33
ratio_of_sd 0.91 0.89 0.89 0.78
monthly_amplitude_ratio_of_means 0.92 0.87 0.86 0.68
ks_mean_on_coarse_res_with_extremes 0.25 0.25 0.24 0.43
qqplot_mae 8.56 11.55 13.07 26.77
acf_mae 0.04 0.03 0.02 0.08
extremogram_mae 0.04 0.04 0.04 0.05

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

cesm2.ssp585

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric naive cnn xgboost lstm
diff_of_means 2.47 4.05 5.03 10.65
ratio_of_sd 0.92 0.89 0.89 0.80
monthly_amplitude_ratio_of_means 0.95 0.89 0.86 0.71
ks_mean_on_coarse_res_with_extremes 0.44 0.29 0.19 0.49
qqplot_mae 7.38 11.01 11.85 24.99
acf_mae 0.04 0.03 0.02 0.08
extremogram_mae 0.04 0.04 0.04 0.04

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

ec_earth3.ssp434

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric naive cnn xgboost lstm
diff_of_means 7.28 10.02 12.41 18.96
ratio_of_sd 0.87 0.84 0.88 0.70
monthly_amplitude_ratio_of_means 0.89 0.81 0.86 0.64
ks_mean_on_coarse_res_with_extremes 0.22 0.18 0.19 0.42
qqplot_mae 14.90 20.84 24.71 39.67
acf_mae 0.05 0.04 0.02 0.06
extremogram_mae 0.05 0.04 0.03 0.06

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

mri_esm2_0.ssp245

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric xgboost naive lstm cnn
diff_of_means −4.18 −7.08 10.78 −12.16
ratio_of_sd 0.96 0.99 0.90 1.04
monthly_amplitude_ratio_of_means 0.84 0.87 0.61 0.82
ks_mean_on_coarse_res_with_extremes 0.23 0.28 0.52 0.46
qqplot_mae 9.39 14.92 23.98 24.21
acf_mae 0.09 0.13 0.17 0.15
extremogram_mae 0.04 0.04 0.05 0.04

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

mri_esm2_0.ssp370

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric xgboost naive cnn lstm
diff_of_means −1.05 −4.53 −10.40 13.85
ratio_of_sd 0.96 1.00 1.03 0.87
monthly_amplitude_ratio_of_means 0.88 0.92 0.86 0.61
ks_mean_on_coarse_res_with_extremes 0.26 0.28 0.49 0.55
qqplot_mae 5.20 10.36 20.70 29.29
acf_mae 0.06 0.11 0.13 0.15
extremogram_mae 0.04 0.03 0.03 0.05

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.

mri_esm2_0.ssp434

Numerical evaluation metrics

  • Each row correspond an evaluation metric
  • Dark blue cell indicates the best model in the row.
  • Light blue cell indicates the second best model in the row.
Metric xgboost naive cnn lstm
diff_of_means −2.87 −5.80 −11.29 11.56
ratio_of_sd 0.96 1.00 1.03 0.88
monthly_amplitude_ratio_of_means 0.87 0.90 0.87 0.62
ks_mean_on_coarse_res_with_extremes 0.25 0.43 0.64 0.77
qqplot_mae 6.72 12.61 22.47 25.55
acf_mae 0.07 0.12 0.13 0.16
extremogram_mae 0.05 0.03 0.05 0.07

Time series of the first few days

Graphical evaluation metrics

  • Observations: cumulative distribution (Detrended QQ), dependency structure (ACF), and extremes (Extremogram, using 0.97 cut off).

  • Daily/Monthly gruped: Maximum hour histogram (only in daily variables), Amplitude distribution.

  • Hourly/Daily grouped: Mean profile per hour within days (or weeks witihin season)

  • In all plots, grey color represent the true ERA5 signal.